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Abstract 



A Piagetian type task. The Island Puzzle, 
was administered to children grades 5 to 12, tc science 
teachers attending a National Science Teachers Association 
Convention, and to college physics teachers attending a 
regional meeting cf the American Association of Physics 
Teachers. The procedures differed from these cf Piaget in 
that the puzzle was administered to a group of subjects and 
the individual responses were written. Subjects 
participating in the study were shewn a map cf four islands 
and a puzzle involving flights from one island tc another 
was presented alcng with clues. Three guestiens were asked 
about whether flights cculd be made between certain 
islands. Answers tc the questions were grouped into six 
different categories. A total of 449 subjects were involved 
in the study. Results cf the study indicate that 
intellectual development in abstract reasoning progresses 
gradually from grade 5 to grades 10 tc 12, where little 
further progress is made and abstract reasoning reaches a 
plateau. The seguence cf six categories established in this 
study appear to be very close to the seguence Piaget and 
his collaborators have found applicable to the tasks 
investigated by them. (ER) 
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The Islands Puzzle (Figure 1) was created as a tool to assess 
abstract reasoning ability. The "clues" given as part of the 
puzzle must be analyzed and used to draw certain conclusions, 
as explained in Figure 1. Note that both an answer and an explana- 
tion of the answer in terms of the clues are required. In this 
way our study is similar to those of PiagetJ Furthermore, we 
have found it most useful to examine and categorize the explanations, 
just as is done by Piaget. Our procedure deviates from that of 
Piaget, however, in that the puzzle is administered to a group of 
subjects and that the individual responses are written. It is 
therefore not possible to investigate a particular subject's think- 
ing beyond the level of the standard questions. Furthermore, we 
have not conducted any longitudinal studies in which the develop- 
ment of a single individual is observed over several years. 

Because the answers are written, the puzzle has been presented 
only to fifth graders and older persons. Phillips who has recent- 
ly reported on a classroom presentation of Piaget-type tasks, was 
able to work with fourth, fifth, and sixth graders and found consid- 
erable agreement between individual and group responses. 



Method of Presentation 



For all groups studied, the task was administered by the authors 
or a specially trained assistant. The subjects were shown a map of 
the four islands (Figure 1), which we designated by the indicated 
names for the children, by letters for the high school students and 
adults. Then the introduction was presented; it was emphasized that 
all plane routes operated in both directions, and that non-stop trips 
or trips with stops or plane-changes were equivalent. No "practice 
questions" or other teaching was provided. The results of a pilot 
study where practice was provided are reported in an Appendix to this 
paper. 
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Figure l. Islands Puzzle 
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Introduction : The puzzle is about four islands in the ocean. 

People have been traveling among these islands by boat for many 
years, but recently an airline started in business. Listen 
carefully to the clues I give you about possible plane trips. 
The trips may be direct or they may include stops on one of 
the islands. When I say a trip is possible, it can be made in 
both directions between the islands. 



I 

This is a map with the four islands, called Bean Island, 
j Bird Island, Fish Island, and Snail Island. You may make 

| notes or marks on your map to help you remember the clues. 

Raise your hand if you have questions about the clues. 



First clue : People can go by plane between Bean and Fish Islands. 



Second clue : People cannot go by plane between Bird and Snail Islands. 

Use these two clues to answer question 1. 

Question 1 : Can people go by plane between Bean and Bird Islands? 

Yes? No? Can't tell from the two clues. 

Explain your answer. 

Third clue : People can go by plane between Bean and Bird Islands. 



Use all three clues to answer questions 2 and 3. Don't change 
your answer to question 1. 



Question 2 : Can people go by plane between Fish and Bird Islands? 

Yes? No? Can't tell from the three clues. 

Explain your answer. 

Question 3 : Can people go by plane between Fish and Snail Islands? 

Yes? N o? Can't tell from the three clues. 

ExplaTrf your answer. 



r; ERIC 




S95VM9B 



msmmmm wmm 




i 



i 




S 



! 




I 

i 

r v 

I 

’ 



| 



\- 



i 









V 




% 



n O 

ERIC 





Most of the groups had answer sheets with the map and the three \ 

question items, but the introduction and clues were always presented J 

orally and clarified with the aid of a blackboard map. The clues \ 

were repeated as often as requested by the subjects. Nevertheless, j 

it was clear from a few of the answers that some subjects were confused j 

by the clues and did not remember the details correctly. The comple- ; 

tion of the puzzle required between ten and fifteen minutes. j 

j 

At this point the reader may wish to attack the Islands Puzzle i 

and record his answers and explanations on a piece of paper. j 



Categorization of Responses 

One subjective aspect of this study is our categorization of a 
subject's explanation of his answers. As we have pointed out, the 
fact that the task was presented to groups prevented detailed explora- 
tion of the reasoning of individuals. Some of the responses are very 
brief and therefore ambiguous; these may represent inadequate reasoning 
or the subject's judgment that the answer is so obvious it could be 
explained briefly. Nevertheless, we have established a sequence of 
six categories which comes very close to the sequence Piaget and his 
collaborators have found applicable to the tasks investigated by 
them. 2 A description of the categories follows. 

Category N: no explanation or statement "I can't explain." 

Category I (pre-logical ) : an explanation which makes no reference 

to the clues and/or introduces new information. Subcategories are 
the mere repetition of the answer to be explained (to #2, "Yes, be- 
cause there are flights"), appeal to the diagram itself (to #2, "No, 
because it is the diagonal" or to #3, "Yes, because it is close"), 
and fanciful stories (to #1, "No, because there is a strong air 
pocket that no one can survive" or to #2, "No, because the plane can 
run out of gas and go down in the water"). 

Category 1 1 a (transition to concrete models): direct appeal to 

or repetition of clues (#1, "No, because you did not say so" or to 
#1 , "Can't tell because you didn't say"). Since all three questions 
require inferences, a direct appeal to the clues does not provide a 
logical justification. 

\ 

Category lib (concrete models): the clues are used to con- 

struct models which are then used to make the predictions. The 
most common model provides for the presence or absence of airport 
facilities on an island, according to whether flights were or 
were not said to reach it (to #1 , "Can't tell, because Bean Island 
has an airport, but Bird Island might or might not have an airpurt"; 
to #2, "Yes, because there must be an airport on Bird Island, so 
the people from Fish Island can get there"; to it 3, "No, Snail must 
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be the one with no airport, so people from Fish Island can't get 
there"). This model-based approach, when correctly used, leads 
to correct answers to all three questions in the problem. It 
assumes information not given in the clues, however, and cannot be 
generalized to solve similar puzzles with different data. 

Category Ilia (transition to abstract logic): logical explana- 

tion to question 2, that Bird Island can certainly be reached from 
Fish Island by way of a stop at Bean Island (to #2, "Yes, Fish to 
Bean to Bird"). Since the logical inference from the two positive 
statements (clues 1 and 3) needed for question 2 is easier, in our 
view, than the use of the negative statement (clue 2), question 2 
does not make maximum demand on the subject's reasoning ability. 

We have therefore classified the logical answer here as being transi- 
tional to the abstract stage, rather than representing attainment 
of the abstract stage. 

Category Illb (abstract logic): logical explanations to 

questions 1 and 3 (to #1, "Can't tell because there is no informa- 
tion linking either Bean or Fish Island with Bird Island"; to #3, 

"No, because a flight between Fish and Snail would make possible 
a route between Bird and Snail via Bean and Fish; this contradicts 
the second clue") . 

It is clear that these six categories must be viewed as tentative 
pending further study, in depth, of the reasoning of individual 
children and adults. The only difficulty we encountered was with 
a large number of explanations of the form "inadequate information" 
to question 1. The response rarely included a comment as to how or 
why the information was inadequate. We did not know whether the 
subject^ actual ly knew the logical reason, or did not and drew the 
conclusions on partially intuitive grounds. We therefore interpreted 
these answers in the light of the explanations offered to questions 2 
and 3. 

The descriptive terms "pre-logical 9 " "concrete models," and 
"abstract logic" are used here to refer to sub-stages in the transition 
from concrete to formal thought, since our observations make this 
refinement necessary. Furthermore, we believe we have found here 
indications of a development that is analogous to the pre-ope rational-- 
concrete operations— formal operations sequence,' in which reference 
to concrete models takes the place of actions on concrete objects. 



Results 

A total of 449 subjects in six distinct groups participated in 
this investigation (Table 1). The groups range from fifth and sixth 
graders to NSTA members at the 1969 convention and college physics 
teachers at a regional meeting of the American Association of Physics 
Teachers. The percentage of subjects in each group placed in each 
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category of explanation is reported in Table 2. For the purposes of 
this table, a subject was placed in the category of his explanations 
if all three fell within the same category. If they varied, and this 
was usually the case, an intermediate category was chosen. Thus, a 
logical answer to #2 (Ilia) and a repetition of clues to #3 (I la) re- 
sulted in the assignment of the subject to category lib. 



Table l. The Subjects 



Subject Group 


Number 


Designation of Group 


Suburban fifth and sixth 
graders 


55 


"5-6" 


Suburban seventh, eighth, 
and ninth graders enrolled 
in a science class 


78 


"7-9" 


Suburban tenth, eleventh, and 
twelfth graders in several 
college preparatory classes 


98 


"10-12" 


Suburban twelfth graders in 
physics classes 


66 


"12P" 


NSTA Convention participants 
at Piaget symposium 


83 


"NSTA" 


American Association of 
Physics Teachers 


69 


"AAPT" 


Total subjects 


449 





It can be seen in the table that there is a gradual progress of 
the group median from category I for the "5-6" group to category I la 
for the "7-9" group and finally category lib for the "10-12" group. 
Curiously enough, there is little further progress even on the part 
of the adult groups. Perhaps a psychological set created by the intro- 
duction is limiting the logical thought. 



Table 2. Evaluation of Explanations (percent) 

Group 



Category 


"5-6" 


"7-9" 


"10-12" 


"12P" 


"NSTA" 


"AAPT 


"■ N 


24 


" ‘3 ' 


1 


9 


10 


0 


I 


40 


39 


9 


15 


4 


3 


1 1 a 


11 


26 


13 


6 


18 


6 


1 1 b 


18 


18 


62 


47 


54 


51 


Ilia 


7 


15 


11 


15 


8 


27 


mb 


0 


0 


3 


8 


6 


13 







More specifically, a large fraction of no explanations (N) by the I 

"5-6" group disappears for the later groups. This is not surprising. 1 

Conversely, answers placed in the category of abstract thought (Illb) ' \ 

are completely absent from the tested samples of fifth to ninth graders » | 

and appear only with high school groups. Since the abstract explana- H 

tion is quite complicated to write down, it is likely that some of ■ 

the younger children would display a better performance in an interview 
compared to a written examination. The combined levels Ilia and Illb 
show surprisingly little variation from group to group, with the "5-6" I 

being somewhat lower than the average and the high school physics group 
being somewhat higher but both at the borderline of statistical signi- 
ficance. Only the college physics teachers show a significantly in- j 

creased percentage in the categories Ilia and Illb. I 

| 

A few interesting results are not revealed by Table 2. The high S 

school groups, for instance, include about 30% model makers (within S 

the scope of category lib), while the "NSTA" group has less than 10% I 

model makers. Almost all the adults answer logically (Ilia) to #2 j 

but very many appeal to the clues directly (Ila) to items #1 and #3. i 

Perhaps the cognitive style, rather than only the intellectual level ] 

of a subject, influences his response. Also, various groups use 1 

different techniques for record keeping. Diagrams are used by some, I 

complete statements by others, abbreviated notes by most adults. ' \ 

i 

Because the total number of correct answers (that is, the ] I 

answer pattern "Can't tell "--"yes "--"no"--which makes optimal use I 

of the information given) is quite small--only 78 out of 449 subjects— 1 

and because the percentage varies only slightly from group to j 

group (from 9% in the "5-6" group to 25% in the "AAPT" group), we j 

have combined the five groups into one and the six categories into 
three. The numbers and percentages of subjects giving correct answers I 

are reported in Table 3. As might be expected, subjects who explain j 

their answers on a higher level of abstraction are much more success- 
ful in obtaining correct answers than the others. From zero correct 1 

answers in the N, I category the percentage rises to 60 in category 
III. On a random basis, one would expect about 3% correct answers 1 

(1 in 27) since there are three questions with three options each. 

Even the level II subjects perform somewhat better than random, but j 

the excess is only at the borderline of statistical significance. j 



Table 3. Correct Answer Patterns 
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Category 


Number of 
subjects 


Number of 
correct answers 


Percentage of 
correct answers 


N, 1 


106 


0 


0 


Ila, lib 


254 


25 


10 


Ilia, Illb 


89 


53 


60 


Total 


449 


78 
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Discussion 



It is clear from this study that intellectual development in 
abstract reasoning, as defined by the "Islands Puzzle," reached a 
plateau in the high school age group and did not progress much 
further. In addition, the plateau is at a disappointingly low 
level. 

The following question of educational policy is thereby raised: 
is it a desirable educational objective that a larger fraction of 
the adult population (other than college physics teachers) should 
be able to answer the Islands Puzzle or an equivalent task on the 
level of abstract thought? We are pleased to be able to state this 
objective in such clear-cut behavioral terms, and we solicit the 
reader's opinion on the matter. If the answer is "yes," then the 
problem becomes one of designing a suitable educational program. 
Unfortunately, even the behavioral statement of the objective does 
not tell us how to proceed, and we again solicit the reader's 
suggestions. 

Even though one can conceive other and better ways of posing 
the logical task than through the Islands Puzzle, we believe that 
this preliminary study calls attention to a neglected area of educa- 
tional research which has significant bearing on large-scale improve- 
ments in scientific literacy. It is furthermore likely that tasks 
can be constructed to assess conservation reasoning, combinatorial 
logic, mathematical proportion, and other components of formal 
thought. What will these reveal about the intellectual level of 
our high school and adult populations? 

We are indebted to Beverly R. Karplus and Margaret A. Karplus 
for assistance in the study. It was supported in part by a grant 
from the National Science Foundation. 
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Appendix 

It is well known that even a brief teaching or "practice" 
exercise improves performance on many tests. Presumably the 
practice furnishes a pattern for the subject's subsequent per- 
formance. Even though more of the subject's ability is there- 
by brought to bear on the test, there is a question of just 
how applicable this ability is to situations where the subject 
is not cued by a practice item. We therefore consider it an 
open question as to whether a performance on the Islands Puzzle 
with or without a practice item is a more satisfactory indi- 
cator of intellectual development. Nevertheless, we have in- 
vestigated the effect of a brief "practice" activity to explore 
this alternative. 

The nature of the Islands Puzzle is such that many differ- 
ent forms of "practice" could improve performance. Since we 
were only concerned with identifying an effect, we chose to 
have the task administrator pose and answer a question analogous 
to puzzle question 3, which was the most difficult one for the 
subjects. The following statement was inserted after the 
introductory paragraph (Figure 1) and before the answer sheets 
with the pictured islands were distributed: 

"I'll now give you one guzzle as an example , and I'll tell 
you my answer to this puzzle % Then, I'll give you another puzzle 
and a piece of paper on which you can write your answers . 

"In my puzzle there one three islands , Island X, Island Y, 
and Island Z . ( Three islands cere drawn in a triangular arrange- 

ment on the chalkboard . During the subsequent explanation , the 
islands mentioned are indicated by hand gestures . ) Now I'll give 
you two clues . The first clue is that people can go by plane 
between Islands X and Y . They can go in both directions , and it 
may be direct or with a stopover . The second clue is that people 
cannot go by plane between Islands X and Z. They cannot go from 
X to Z or Z to X either directly or by stopping somewhere . 

"There is only one question left; what about plane trips 
between Islands Y and Z? Can people go by plane between Islands Y 
and Z? You could answer ' Yes' or 'No' or ' Can't tell from gust 
the two clues t ' Now I'll tell you my answer , and then I'll explain 
the reason for my answer by using the two clues . 

"My answer is .'No. ' And the explanation goes like this: 
according to the first clue , people can go by plane from X to Y. 
Now, if there are planes to go from Y to Z, then people can go 
from X to Y and catch another plane from Y to Z. That way they 
could get from X to Z. But this contradicts the second clue, which 
said that people could not_ go by plane between X and Z. Therefore , 
people cannot go by plane between Y and Z." 
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Three groups of students, each in the same school as the 
corresponding group identified in Table 1, worked on the Islands 
Puzzle after the practice exercise. The results of their work 
are presented in Table 4 and may be compared with those for the 
first three groups in Table 2. 



Table 4. 

Category 


Evaluation of Explanations with Practice (percent) 

Group 

"5-6" "7-9" "10-12" 




ii 


(N = 139) 


(N = 114) 


N 


5 


2 


2 


I 


20 


10 


11 


I la 


30 


24 


16 


lib 


45 


45 


33 


Ilia 


0 


13 


24 


mb 


0 


6 


15 



It is clear that all three groups show substantial gains. 
The number of students in category I has decreased dramatically. 
The median has shifted upward for each group, to category I la 
for the "5-6" group, to category lib for the "7-9" group, and 
above category" I lb for the "10-12" group. The single most 
noticeable change in the individual answers is the appearance 
of hypotheti co-deductive statements (to #1, "Yes, if Bean can 
go to Fish, then I think it could go to Bird") even though 
the logic of these statements is frequently incorrect. Still, 
the steady advance with grade level from pre-logical to abstract 
logical thinking is very similar to the advance in the puzzle 
without practice. As might be expected, the number of students 
with a correct answer pattern has increased (Table 5, compare 
with Table 3), but is still far from complete. 



Table 5. Correct Answer Patterns with Practice 



Category 


Number of 
subjects 


Number of 
correct answers 


Percentage of 
correct answers 


N, I 


42 


2 


5 


Ila, lib 


185 


55 


30 


Ilia, Illb 


70 


51 


79 
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Total 297 108 
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One last point should be made. The students reacted to the 
practice puzzle with subdued but noticeable signs of dismay. 

This had not been the case at all when no practice was provided. 
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